Papers with transformer-specific dropout methods
LoRA Meets Dropout under a Unified Framework (2024.findings-acl)
Copied to clipboard
| Challenge: | Parameter-efficientfinetuning (PEFT) has gained popularity as a lightweight approach for model customization. |
| Approach: | They propose a parameter-efficient dropout method that is overfitting-prone and parameter-freezed. |
| Outcome: | The proposed method is superior to existing methods and compares with transformer-specific methods. |